77.20 


Decisions  by  Director,  NIH,  £ 
on  Recommendations  of 
Grants  Peer  Review  Study  Team 

February  8,1978 
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SUMMARY  OF  DECISIONS  BY  DIRECTOR,  NIH 
ON  RECOMMENDATIONS  OF  GRANTS  PEER  REVIEW  STUDY  TEAM 


Dr.  Donald  S.  Fredrickson,  Director  of  the  National  Institutes  of 
Health,  announced  today  that  he  has  approved  33  of  the  recommenda¬ 
tions  submitted  to  him  by  the  NIH  Grants  Peer  Review  Study  Team 
for  the  improvement  of  the  Agency's  Grants  Review  Process. 
Additionally,  nine  of  the  Team’s  proposals  were  approved  with 
minor  modifications. 

The  NIH  Deputy  Director,  Dr.  Thomas  E.  Malone,  was  given  the 
responsibility  to  carry  out  the  approved  recommendations  and  to 
arrange  for  further  studies  of  19  of  the  proposals  deferred 
pending  additional  examination  and  discussion.  Three  of  the 
Study  Team's  recommendations  were  rejected  and  five  required  no 
action  by  the  Director. 

The  12-member  NIH  Grants  Peer  Review  Study  Team,  headed  by  Dr. 

Ruth  L.  Kirschstein,  Director  of  the  National  Institute  of 
General  Medical  Sciences,  conducted  a  searching  review  of  the 
system  used  by  the  National  Institutes  of  Health  for  evaluating 
the  more  than  15,000  research  and  training  proposals  the  Agency 
receives  each  year. 

In  the  course  of  the  study,  the  Team  held  public  hearings  in 
Chicago,  San  Francisco  and  Bethesda  and  actively  sought  written 
comments  from  the  present  and  former  non-Federal  members  of 
review  bodies,  as  well  as  from  applicants,  grantee  institutions 
and  the  general  public. 

One  set  of  the  recommendations  approved  by  Dr.  Fredrickson 
was  designed  to  improve  communication  with  applicants,  and 
would  require  NIH  Advisory  Councils  and  Boards  to  provide  all 
applicants  with  the  complete  Summary  Statements  or  critiques  of 
their  proposals,  including  assigned  priority  scores,  promptly 
after  final  action  is  taken  on  them. 

Another  cluster  of  the  proposals  adopted  by  the  Director  is 
aimed  at  opening  the  process  for  nominating  and  selecting  non- 
Federal  advisors  for  service  on  councils  and  initial  review 
groups.  It  calls  for  widespread  publication  of  information  about 
the  peer  review  system  itself  with  periodic  general  invitations 
for  nominations  of  potential  council  or  review  group  members.  A 
central  list  of  nominees  will  be  maintained  for  referral  to  the 
various  advisory  groups  as  replacements  for  members  whose  terms 
expire . 
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The  approved  recommendations  would  also  require  that  the  member¬ 
ship  of  special  review  groups  be  reported  to  the  applicant  whose 
proposal  they  review  and  that  the  names  and  qualifications  of  all 
appointed  council  members  be  published  in  the  NIH  Guide  to 
Grants  and  Contracts. 

Several  of  the  recommendations  approved  by  Dr.  Fredrickson 
prescribed  means  for  tightening  procedures  to  avoid  apparent  or 
real  conflicts  of  interest  involving  members  of  review  groups. 

Dr.  Fredrickson  withheld  his  approval  pending  further  study  of 
a  group  of  recommendations  to  establish  a  formal  grants  peer 
review  appeals  system  including  the  appointment  of  an  ombudsman. 

In  deferring  his  decision  on  implementation  of  the  appeals 
system,  the  Director  noted  that  setting  up  such  a  system  would 
entail  a  number  of  legal,  financial  and  personnel  resource 
questions  which  must  be  further  evaluated. 

The  Study  Team  had  recommended  that  applicants  be  required  to 
identify  proposals  they  consider  to  be  "especially  innovative" 
and  to  support  that  contention  in  detail.  In  the  same  vein, 
the  Team  also  proposed  that  NIH  Initial  Review  Groups  be  encouraged 
to  prepare  special  statements  regarding  "innovative  aspects"  of 
proposals,  and  to  flag  such  proposals  for  special  consideration. 
Both  recommendations  were  rejected  by  Dr.  Fredrickson  on  the 
ground  that  researchers  generally  consider  their  work  to  be 
innovative  and  that  review  groups  especially  seek  this  quality 
in  their  analyses.  Rather  than  create  a  special  class  of 
proposals,  he  recommends  that  review  group  members  be  encouraged 
and  reminded  to  give  due  weight  to  "innovative"  aspects  of  all 
applications  in  setting  their  priority  scores. 

Dr.  Fredrickson  also  rejected  the  recommendation  that  NIH  seek 
to  have  the  authority  for  appointments  to  advisory  councils  and 
boards  delegated  from  the  Secretary,  HEW,  to  the  Assistant 
Secretary  for  Health. 

Following  the  submission  of  the  GPRST  Report  and  Recommendations 
to  the  Director  in  January  1977,  he  appointed  a  small  working 
group  of  senior  staff  members  and  program  heads  to  examine  each 
recommendation  in  the  light  of  its  legal  and  operational  implica¬ 
tions,  and  to  communicate  with  each  Bureau,  Institute  and 
Division  regarding  the  GPRST  proposals.  The  working  group 
assembled  BID  comments  on  the  recommendations,  and  again  circulated 
them  for  further  comment.  The  final  BID  comments  and  consensus 
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of  the  members  of  the  working  group  together  with  proposals  for 
action  were  then  submitted  to  the  Director  for  his  decisions. 

Attached  is  a  listing  of  the  Study  Team's  69  recommendations 
together  with  the  Director's  decisions  and  comments  regarding 
them . 


Attachment 


DECISIONS  BY  DIRECTOR,  NIH  ON  RECOMMENDATIONS 


OF  GRANTS  PEER  REVIEW  STUDY  TEAM 


Recommendations  #1  and  #9A  were  considered  together: 

RECOMMENDATION  # 1  --  (  # 1  page  81  of  the  GPRST  Report) 

That ,  in  order  to  make  the  reviewer  selection  process  better  and 
more  widely  understood  and  to  regularize  the  ways  in  which  the 
scientific  and  lay  communities  contribute  information  in  support 
of  it3  the  NIE  should  publish  periodically  (e.g.3  annually) 
an  announcement  which  calls  attention  to  the  upcoming  vacancies 
on  IRGs  associated  with  the  Grants  Peer  Review  System  and  which 
invites  suggestions  regarding  candidates  for  specific  IRGs. 

RECOMMENDATION  #9 A  --  (#2  page  94  of  the  GPRST  Report) 

That  the  procedure  for  nominating  and  selecting  Council  members 
be  more  open.  Scheduled  Council  vacancies  should  be  announced 
and  published  early  in  the  fiscal  year.  The  announcement  should 
specify  the  Council  involved 3  the  criteria  for  the  selection  of 
Council  member s 3  their  duties 3  and  terms  of  appointment . 

DECISION 


The  process  of  selecting  reviewers  for  NIH  grant  proposals 
should  be  open.  In  order  to  broaden  the  approach  recommended 
by  the  Study  Team  we  will  develop  a  reservoir  of  individuals 
qualified  to  serve  on  advisory  groups  by: 

•  publishing  a  description  of  the  NIH  Peer  Review  System 
as  an  issue  of  the  NIH  Guide  to  Grants  and  Contracts , 

as  well  as  in  at  least  one  widely  read  scientific  journal. 

•  publishing  periodically  descriptions  of  the  Peer  Review 
System,  listing  current  advisory  groups,  detailing  the 
selection  process  ,  and  inviting  nominations  of  potential 
members.  Such  nominations  will  be  collected  centrally 
and  subsequently  referred  to  the  appropriate  advisory 
group.  In  this  way  the  NIH  will  develop  a  reservoir 

of  potential  consultants  from  which  replacements  may  be 
selected  for  those  reviewers  whose  terms  are  expiring. 
Every  effort  must  be  made  to  seek  scientists  who  are 
women,  ethnic  minorities  and  of  a  younger  age  and  have 
skills  consistent  with  the  demanding  standards  of  the 
review  process. 


2 


RECOMMENDATION  # 2  --  (# 2  page  82  of  the  GPRST  Report) 

That j  in  order  to  provide  for  additional  advice  beyond,  that  given 
by  Initial  Reveiw  Group  member  s  3  particularly  in  regard,  to  those 
research  grant  applications  in  which  new  scientific  fields  o.re 
to  be  explored,  or  unique  procedures  are  to  be  developed 3  the  NIB 
should,  implement  a.  formal  procedure  whereby  an  applicant  may 
identify  those  particular  aspects  of  the  proposed  research  which, 
are  considered,  to  be  unique  and  to  suggest  possible  consultants 
who  are  considered  to  be  leaders  in  and  currently  knowledgeable 
about  his/her  area  of  research.  However  3  the  decision  as  to 
whether  to  seek  advice  from  the  suggested  expert ( s )  or  any  other 
expert ( s )  must  remain  that  of  the  NIH. 

DECISION 

It  is  recognized  that  most  applicant-investigators  consider 
their  research  to  be  unique.  However,  the  recommendation  that 
applicants  may  suggest  their  own  reviewers  could  result  in  the 
creation  of  apparent  or  real  conflicts  of  interest  whose  avoidance 
would  add  special  administrative  problems  to  an  already  over¬ 
burdened  system. 

Special  efforts  must  be  made  by  reviewers  and  staff  to  identify 
unique  or  unorthodox  research.  The  need  to  be  alert  to  unique 
ideas  should  be  stressed  in  the  orientation  of  new  members  of 
initial  review  groups  and  councils  and  in  the  Guide  for  new 
members.  Executive  Secretaries  and  staff  should  be  especially 
attentive  to  this  need. 

RECOMMENDATION  # 3  --  (#3  page  83  of  the  GPRST  Report) 

That  the  current  practice  of  consultation  with  present  3  retiring  3 
and  former  IRG  members  concerning  possible  new  members  should 
continue  to  be  but  one  of  a  series  of  steps  by  which  advice 
concerning  potential  IRG  members  is  sought  from  many  sources . 

DECISION 


Concur . 

RECOMMENDATION  # 4  --  (#4  page  83  of  the  GPRST  Report) 

That  the  current  practice  of  providing  only  one  nomination  for  each 
IRG  vacancy  should  be  retained .  Furthermore3  advice  regarding 
such  nominations  to  DRG  Initial  Review  Groups  should  be  sought 
from  knowledgeable  and  interested  staffs  of  the  various  BIDs 
responsible  for  the  administration  of  grants  in  the  particular 
biomedical  scientific  areas. 
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DECISION 


Concur . 

RECOMMENDATION  #  5  --  f# 1  on  page  88  of  the  GPRST  Report) 

That  it  is  essential  that  NIH  continue  to  have  the  flexibility 
and  opportunity  to  establish  special  Initial  Review  Groups. 

DECISION 

Concur . 

RECOMMENDATION  # 6  --  (#2  on  page  88  of  the  GPRST  Report) 

That  NIH  and  HEW  should  oppose  legislation  regarding  ad_  hoc  groups. 

DECISION 


Concur . 

RECOMMENDATION  #7  --  (#3  on  page  89  of  the  GPRST  Report) 

That  the  roster  of  consultants  to  be  included  on  a  special  Initial 
Reveiw  Group  be  submitted  to  all  investigators . 

DECISION 


All  consultants  serving  in  a  Special  Initial  Review  Group 
should  be  listed  in  the  Summary  Statement  transmitted  to  the 
applicant-investigator.  (See  Rec.  #53  below.) 

RECOMMENDATION  # 8  --  (#1  page  94  of  the  GPRST  Report) 

That  NIH  should  seek  to  have  the  authority  for  selection  and 
appointment  of  members  of  Advisory  Councils /Boards  delegated  to 
the  Assistant  Secretary  for  Health3  DHEW . 

DECISION 

I  believe  that  it  is  appropriate  for  the  Secretary  of  Health, 
Education,  and  Welfare  to  retain  authority  to  select  and  appoint 
members  of  Advisory  Councils .  Therefore  I  do  not  concur  with 
the  Study  Team's  recommendation. 

RECOMMENDATION  #9A  was  considered  with  Recommendation  1,  page  1. 
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RECOMMENDATION  #9B  --  (# 2  page  94  of  the  GPRST  Report) 

The  names  and  qualifications  of  the  selected  Council  members 
should  be  published . 

DECISION 

I  concur  and  suggest  that  the  names  and  qualifications  of 
selected  Council  members  be  published  in  the  NIH  Guide  to  Grants 
and  Contracts . 

RECOMMENDATION  # 10  --  (#3  page  95  of  the  GPRST  Report) 

That ,  when  a  selection  has  been  made  for  a  Council  vacancy (ies) 
other  than  from  nominations  submitted  by  the  Director ,  NIH,  the 
appointment  should  not  be  made  final  until  the  Director ,  NIH,  has 
had  the  opportunity  to  comment  on  the  selection. 

DECISION 


Concur . 

RECOMMENDATION  # 11  --  (#1  page  100  of  the  GPRST  Report) 

That  NIH  develop  detailed  supplemental  instructions  to  be  sent  to 
members  along  with  the  Form  HEW  474,  emphasizing  that  employment 
includes,  for  example ,  foreign  employment,  summer  and  part-time 
employment,  membership  on  advisory  boards  of  organizations ,  and 
consultant  appointments ;  and  that  relevant  financial  interests 
include  such  things  as  royalty  agreements  with  or  stock  ownership 
in  domestic  or  foreign  biomedical  research  organizations ,  companies 
engaged  primarily  in  providing  service  to  such  organizations , 
pharmaceutical  companies,  and  other  companies  involved  in  drug 
research. 

DECISION 


Concur . 

RECOMMENDATION  # 12  --  (#2  page  101  of  the  GPRST  Report) 

That  NIH  adopt  a  procedure  under  which  Form  HEW  474  is  returned  to 
a  member  as  incomplete ,  where  such  member  makes  no  entry  in  a 
section  of  the  form  but  does  not  write  "none"  or  some  equivalent 
in  the  section. 

DECISION 


Concur . 
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The  following  three  recommendations  were  considered  together: 

RECOMMENDATION  #15  --  (#3  page  101  of  the  GPRST  Report) 

That  the  Executive  Secretaries  of  IRGs  he  given  access  to  the 
Forms  HEW  474  of  member s 3  as  needed. 

RECOMMENDATION  #14  --  (# 4  page  101  of  the  GPRST  Report) 

That  review  of  Forms  HEW  474,  at  least  annually 3  by  each  Executive 
Secretary  be  mandated  throughout  NIH. 

RECOMMENDATION  #15  --  0)5  page  101  of  the  GPRST  Report) 

That  all  Executive  Secretaries  be  required  periodically  to  attend 
training  sessions  on  evaluation  of  conflict-of-interest  situations . 

DECISION 

At  this  time  the  recommended  procedures  are  carried  out  in 
some  components  of  NIH  and  not  in  others .  Current  practices 
should  be  reviewed  and  uniform  procedures  be  developed  covering 
all  members  of  initial  review  groups  and  advisory  councils. 

RECOMMENDATION  #16  --  0)1  page  102  of  the  GPRST  Report) 

That  all  initial  invitations  to  serve  on  advisory  committees  make 
it  explicit  that  final  appointment  is  contingent  upon  review  of 
the  completed  Form  HEW  474  for  conflicts  of  interest,  or  potential 
conflicts  of  interest ,  and  that  the  new  member  be  formally  notified 
of  the  appointment  after  the  appointment  process  (including 
review  of  the  Form  HEW  474)  has  been  completed . 

DECISION 


Concur . 

RECOMMENDATION  #17  --  0)2  page  102  of  the  GPRST  Report) 

That  the  names  of  new  members  of  advisory  committees  not  be 
released  to  the  news  media  or  other  members  of  the  public  until 
such  time  as  final  appointment  has  occurred  after  completion  of 
the  Forms  HEW  474. 

DECISION 


Concur . 
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RECOMMENDATION  # 18  --  ((#111  page  105  of  the  GPRST  Report) 

That  employees  of  "for-profit"  organizations  be  eligible  for 
membership  on  all  Initial  Reveiw  Groups  considering  grant 
applications  (including  National  Research  Service  Award 
applications) 3  and  that  the  selection  of  these  individuals  be 
based  on  the  scientific  proficiency  necessary  to  fill  the  various 
needs  of  specific  committees .  The  basis  for  selection  of  such 
scientists  shall  be  the  same  as  for  those  employ ed  by  " non-profit " 
organizations . 

DECISION 


Current  NIH  practices  are  in  accord  with  the  Study  Team's 
recommendation.  No  further  action  is  required. 

RECOMMENDATION  if  19  --  (#111  page  105  of  the  GPRST  Report) 

That  the  conflict  of  interest  statements  (HEW  Form  4 7 4 ) submitted 
by  employees  of  pro  fit-making  organizations  nominated  as  consultants 
must  be  reviewed  by  the  agency  head3  who  must  be  satisfied  that 
each  nominee  can  serve  on  the  specific  committee  for  which  such 
person  has  been  proposed  without  being  in  violation  of  the  conflict 
of  interest  statutes . 

DECISION 


Concur . 

RECOMMENDATION  #20  --  (#IV  page  105  of  the  GPRST  Report) 

The  criteria  for  selection  of  IRG  members  from  "for-profit" 
organizations  should  be  no  different  than  those  for  selection 
of  persons  from  " non-profit "  organizations . 

DECISION 


No  action  required.  See  Rec.  #18  above. 

RECOMMENDATION  #21  --  (Page  116  of  the  GPRST  Report) 

That  the  PHS  Act  be  amended  to  provide  statutory  exemption  from 
the  requirements  of  the  Freedom  of  Information  Act  for  disclosure 
of  research  designs  and  protocols  presented  in  grant  applications . 

DECISION 


Concur . 
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RECOMMENDATION  ft  2 2  --  (Page  119  of  the  GPRST  Report) 

That  NIH  and  HEW  present  specific  written  opposition  to  any 
legislative  proposal  extending  the  Federal  Advisory  Committee 
Act  to  ad_  hoc  or  Special  Review  Groups ,  based  on  the  need  to 
prevent  conflicts  of  interest  in  the  peer  review  of  grant 
applications  and  the  need  to  retain  flexibility  in  meeting  short- 
time  workload  increases  in  specialized  areas. 

DECISION 


The  amendments  to  the  Federal  Advisory  Committee  Act  intro¬ 
duced  in  the  95th  Congress  do  not  contain  any  provision  relating 
to  ad  hoc  or  special  review  groups.  No  action  is  required  at 
this  time.  However,  the  NIH  should  oppose  any  such  legislative 
proposal  when  offered. 

Recommendations  23  and  53  were  considered  together: 

RECOMMENDATION  #23  --  (Page  123  of  the  GPRST  Report) 

That  the  Summary  Statements  be  made  available  upon  request  to 
the  principal  investigators  3  including  the  priority  scores. 

RECOMMENDATION  # 53  --  (# 1  page  178  of  the  GPRST  Report) 

That 3  as  soon  as  practical  after  a  BID  ’ s  National  Advisory 
C ouncil / Board  completes  the  review  of  a  grant  application,  the 
BID  should  routinely  send  the  Summary  Statement,  with  the  priority 
score  display ed 3  to  the  principal  investigator .  Because  Summary 
Statements  frequently  contain  forthright  and  critical  comments 
concerning  the  principal  investigators 3  this  recommendation 
is  made  contingent  upon  the  understanding  that  these  documents 
will  be  released  by  NIH  only  to  these  individuals.  If  it  is 
determined  that  such  NIH-initiated  release  of  Summary  Statements 
could  reasonably  lead  to  a  requirement  that  NIH  make  these  documents 
available  also  to  applicant  institutions  or  others,  then  the  study 
team  recommends  that  Summary  Statements  with  priority  scores 
displayed  be  released  only  under  Privacy  Act  procedures,  i.e. , 
upon  receipt  of  an  appropriate  request  from  the  individual  concerned . 

DECISION 


Recommendation  23  (See  also  Decisions  regarding  Rees.  #53  and 

#54) 


Concur  with  recommendation  to  include  priority  score  with 
Summary  Statements  sent  to  Principal  Investigator  subsequent  to 
final  action  by  National  Advisory  Council/Board.  When  Summary 
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Statements  are  provided  to  Principal  Investigators  prior  to 
final  action,  priority  score  will  be  deleted. 

DECISION:  Recommendation  #53 


Concur,  with  the  condition  that  an 
with  each  Summary  Statement  concerning 
and  normalized  scores  as  a  substantial 
determinant  of  the  funding  decision. 


explanation  be  transmitted 
the  significance  of  raw 
element,  but  not  the  sole 


RECOMMENDATION  # 24  --  (Rage  123  of  the  GPRST  Report) 


That  in  releasing  reviewers  '  opinions  under  the  Privacy  Act,  an 
adequate  legal  basis  be  established  for  protecting  the  reviewers  1 
anomymity  either  through  reinterpretation  of  existing  law ,  if 
reasonably  possible,  or  enactment  of  new  legislation . 


DECISION 


Concur . 

RECOMMENDATION  # 25  --  (Page  124  of  the  GPRST  Report) 

That  those  portions  of  the  meetings  of  advisory  groups  which 
involve  the  review  of  grant  applications  continue  to  be  closed 
to  the  public  (including  those  submitting  applications) ,  either 
under  current  exemptions  to  the  open-meeting  requirement  or 
through  a  statutory  amendment . 

DECISION 


Concur . 

RECOMMENDATION  # 26  --  (Page  124  of  the  GPRST  Report) 

That,  notwithstanding  proposals  which  would  require  public 
representation  on  all  committees,  the  current  system  of  dual 
review  be  preserved,  with  grant  applications  being  reviewed  first 
by  Initial  Review  Groups  consisting  solely  of  scientific  and 
technical  experts  and  then  by  National  Advisory  Councils  and 
Boards  which  include  representatives  of  the  public. 

DECISION 


Concur . 
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RECOMMENDATION  # 2  7  --  (Rage  125  of  the  GPRST  Report) 

That  the  NIH  and  HEW  establish  a  mechanism  for  special 3  periodic 
assessment  of  the  impact  of  this  new  legislation  on  the  quality 
of  grant  applications  and  on  the  quality  of  peer  review  of  such 
applications  and  that  such  assessment  be  reported  to  the  legis¬ 
lative  and  executive  branches  of  the  government . 

(This  recommendation  concerns  the  Privacy  Act,  the  amended 
Freedom  of  Information  Act  and  the  Federal  Advisory  Committee 
Act .  ) 

DECISION 

I  concur  in  principle  with  the  Study  Team's  recommendation 
that  assessment  be  made  of  the  effect  of  the  "Sunshine  laws" 
on  the  quality  of  applications  and  their  review.  However, 
implementation  of  the  recommendation  will  require  first  the 
development  of  acceptable  study  protocols. 

RECOMMENDATION  # 28  --  (#1  page  152  of  the  GPRST  Report) 

That  the  Director3  NIH3  take  immediate  steps  to  limit  the  workload 
of  all  Initial  Grant  Review  Groups  to  a  level  which  is  compatible 
with  maintaining  the  high  quality  of  peer  review. 

DECISION 


Concur . 

RECOMMENDATION  #29  --  (#2  page  152  of  the  GPRST  Report) 

That  authority  to  establish  or  discontinue  Initial  Grant  Review 
Groups3  as  the  peer  review  workload  dictates 3  should  be  delegated 
to  the  Director3  NIH. 

DECISION 


Concur . 

RECOMMENDATION  fl 50  --  (#5  page  152  of  the  GPRST  Report) 

That  additional  resources  be  provided  for  peer  review  of  grant 
applications  where  acceptable  alternative  approaches  to  reduction 
of  workload  will  not  permanently  and  effectively  resolve  long¬ 
standing  review  overloads. 

DECISION 


Concur . 
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RECOMMENDATION  ft  31  --  (ft4  page  133  of  the  GPRST  Report) 

That  the  Director 3  NIH3  establish  a  permanent  mechanism  to 
determine  and  ensure  an  appropriate  ceiling  or  maximum  workload 
for  each  NIH  Initial  Grant  Review  Group. 

DECISION 


Concur . 

RECOMMENDATION  ft32  --  (#5  page  133  of  the  GPRST  Report) 

That  the  NIH  consider  whether  there  are  peer  review  services  to 
NIH  staff,  advisory  councils  3  or  applicants  which  might  be 
curtailed  if  all  other  workload  reduction  alternatives  should  fail. 

DECISION 


Concur . 

RECOMMENDATION  ft33  --  (ftl  page  146  of  the  GPRST  Report) 

That  the  proposed  peer  review  regulations  be  finalized  as  soon 
as  possible. 

DECISION 


Proposed  peer  review  regulations  have  been  submitted  for 
Departmental  review  and  when  approved  will  be  published  in  the 
Federal  Register . 

RECOMMENDATION  #34  --  (ft 2  page  146  of  the  GPRST  Report) 

That  NIH  standards  and  guidelines  be  prepared  or  revised  as  soon 
as  possible  after  consideration  and  evaluation  of  recommendations 
made  in  the  study  by  the  Executive  Secretaries  Review  Activities 
Committee . 

DECISION 


The  Executive  Secretaries  Review  Advisory  Committee  is  con¬ 
ducting  a  study  of  all  policies  and  procedures  developed  by 
NIH  and  its  Bureaus,  Institutes  and  Divisions  for  the  review 
of  grant  applications.  The  committee  has  been  asked  to  recommend 
agency-wide  guidelines  and  standards.  I  concur  with  the  Study 
Team  recommendation  regarding  early  consideration  of  such  committee 
recommendations . 
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RECOMMENDATION  #35  --  (#3  page  146  of  the  GPRST  Report) 

That,  in  order  to  maintain  and  improve  the  level  of  excellence  of  the 
grants  peer  review  system  at  NIH,  a  personnel  position  should  he 
established  within  the  Office  of  Extramural  Research  and  Training 
to  provide  for  quality  assurance  of  the  system.  Such  a  person 
should  have  staff  responsibility  for  matters  pertaining  to  the 
formulation ,  development ,  interpretation ,  and  application  of 
policies  relating  to  the  assignment  and  overall  scientific  and 
technical  review  of  research  grants  and  manpower  development 
programs . 

DECISION 

The  Study  Team  correctly  recognized  the  importance  of  the 
function  described.  However,  most  of  them  are  now  part  of  the 
responsibility  of  the  Office  of  the  Associate  Director  for 
Scientific  Review,  DRG.  Prior  to  a  decision  about  establishing 
a  new  position  further  discussion  and  examination  is  necessary. 

RECOMMENDATION  #36  --  (#4  page  146  of  the  GPRST  Report) 

That  training  curricula  be  developed  by  NIH  for  extramural  program 
and  review  staff  in  order  to  provide  orientation  and  to  refresh 
and  reiterate  principles  concerning  the  philosophy ,  objectives, 
and  procedures  for  peer  review. 

DECISION 


Concur . 

RECOMMENDATION  #37  --  (#5  page  146  of  the  GPRST  Report) 

That  open  forums  and  workshops  for  program  and  review  staff  be 
established  on  a  continuing  basis  so  as  to  encourage  and  improve 
exchange  and  communication  of  ideas  concerning  issues  relevant 
to  the  peer  review  system. 

DECISION 


Concur . 

RECOMMENDATION  #38  --  (#6  page  146  of  the  GPRST  Report) 

That  the  Director ,  NIH,  the  BID  Directors ,  and  their  staffs  hold 
orientation  sessions  annua lly  for  all  IRG  and  Council  members , 
particularly  new  members,  in  order  to  place  peer  review  in 
perspective  and  to  inform  reviewers  of  their  functions ,  duties, 
and  responsibilities . 
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DECISION 


Adequate  orientation  of  IRG  and  council  members  is  essential. 

I  have  adopted  the  following  alternative  to  the  Study  Group's 
recommendation  to  accomplish  that  purpose  and  at  the  same  time 
conserve  time  and  effort  of  NIH  officials  and  advisors  --  ("That 
the  responsible  officials  and  staffs  of  the  various  initial  review 
groups  or  Advisory  Councils/Boards  hold  annual  orientation  sessions 
for  their  new  members  in  order  to  place  peer  review  in  perspective 
and  to  inform  reviewers  of  their  functions,  duties  and 
responsibilities . ") 

RECOMMENDATION  #39  --  ( #A  page  160  of  the  GPRST  Report) 

That  it  be  recognized  as  an  NIH  policy  position  that  the  use  of 
business  management  consultants  as  reviewers  is  a  necessary 
adjunct  to  the  scientific  review  of  the  large 3  complex  and 
multifaceted  program  projects 3  centers 3  or  those  grants  involving 
consortium  arrangements . 

DECISION 


Concur . 

RECOMMENDATION  #40  --  (#B  page  160  of  the  GPRST  Report) 

/ 

That  the  principal  criteria  used  to  determine  the  need  for  business 
management  consultant  support  be  related  to  the  organizational  or 
administrative  complexity  existing  in  any  particular  grant 
application . 

DECISION 


Concur . 

RECOMMENDATION  # 41  --  (#C  page  160  of  the  GPRST  Report) 

That }  considering  the  nature  of  the  traditional  investigator- 
initiated  research  project  application  and  its  review ,  it  be 
recognized  that  the  use  of  business  management  consultants  would 
not  usually  be  considered  necessary  or  appropriate. 

DECISION 


Concur . 

RECOMMENDATION  # 42  --  (#D  page  161  of  the  GPRST  Report) 

That  NIH  develop  a  policy  issuance  concerning  the  role  of  and  need  for 
assessment  of  business  management  in  the  review  of  the  large  program 
project  and  center  grant  applications . 


DECISION 
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Concur . 


RECOMMENDATION  #4  3  --  (#E  page  161  of  the  GPRST  Report ) 

That  NIH  develop  uniform  guidelines  to  he  followed  hy  the 
business  management  consultants  in  their  roles  as  members  of 
project  site  visit  teams  and  advisory  groups. 

DECISION 


Concur . 

The  following  two  recommendations  were  considered  together: 

RECOMMENDATION  #44  --  ( #F  page  163  of  the  GPRST  Report) 

That  business  management  consultants  should  contribute  to  over-all 
recommendations  on  project  site  visits  and  at  Initial  Review 
Group  meetings ,  but  they  should  not  vote  on  " approval  vs  disapproval" 
or  give  a  priority  rating  unless,  in  the  judgment  of  the  Executive 
Secretary,  the  type  of  project  includes  significant  non- scientific 
aspects  clearly  within  their  professional  expertise . 

RECOMMENDATION  #45  --  (#G  page  163  of  the  GPRST  Report) 

That  business  management  consultants  should  prepare  a  specific 
portion  of  project  site  visit  reports  and/or  IRG  Summary 
Statements  which  should  be  recognized  as  separate  from  scientific 
review  and  evaluation,  but  should  be  carefully  considered  in 
reaching  recommendations  for  each  project. 

DECISION 


The  issues  raised  by  these  recommendations  have  been  discussed 
at  length  and  inconclusively  by  Institute  Directors  and  other 
reviewers  of  the  Study  Team  report.  Action  will  be  deferred  on 
these  recommendations  pending  further  examination  and  discussion. 

RECOMMENDATION  # 46  --  (#H  page  164  of  the  GPRST  Report) 

That  procedures  be  developed  which  would  allow  the  reports  of  the 
business  management  consultants ,  including  specific  recommendations 
for  management  improvement ,  to  be  made  available  to  the  applicant 
institution  by  the  BID  at  an  appropriate  time  after  completion  of 
the  review. 

DECISION 


Reports  of  business  management  consultants ,  including  specific 
recommendations  for  management  practices  germane  to  the  appli¬ 
cation  under  consideration  should  be  incorporated  in  the  Summary 
Statement . 
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RECOMMENDATION  #47  --  (#1  -page  164  of  the  GPRST  Report) 

That  grants  management  officers  of  the  BIDs  participate  with  the 
IRG  Executive  Secretaries  in  the  determination  of  a  need  for 3 
and  in  selection  o f 3  business  management  consultants  fo r  the 
project  site  visit  team  involvement . 

DECISION 


Concur . 

RECOMMENDATION  #48  --  (#J  page  165  of  the  GPRST  Report) 

That 3  whenever  possible s  BID  grants  management  staff  members 
should  accompany  the  site  visit  teams  on  the  reviews  of  the 
complex  projects 3  serving  as  staff  resources  only. 

DECISION 


Concur,  but  amended  as  follows  —  "That,  whenever  appropriate, 

BID  grants  management  staff  members  should  accompany  the  site 
visit  teams  on  the  reviews  of  complex  projects  serving  as  staff 
resources  only." 

RECOMMENDATION  # 49  --  (# 1  page  171  of  the  GPRST  Report) 

That  NIH  require  the  app licant  to  identify  and  support  in  detail 
the  contention  that  the  research  project  being  proposed  is  especially 
innovative . 

DECISION 


This  recommendation  places  an  unnecessary  obligation  on  the 
applicant-investigator  who  would  be  required  to  amplify  justifica¬ 
tion  of  his  proposal  as  "especially  innovative"  or  tacitly  accept 
its  designation  as  routine  or  pedestrian.  The  peer  review  system 
must  remain  alert  to  innovation.  (See  Rec.  #2  above).  The  Study 
Team's  purpose  in  making  this  recommendation  is  understood,  but  I 
do  not  believe  its  implementation  would  be  productive  and  therefore 
do  not  concur  with  it.  The  approach  proposed  in  recommendation  50 
appears  to  be  much  sounder. 

RECOMMENDATION  #50  --  (#2  page  171  of  the  GPRST  Report) 

That  Initial  Review  Groups  be  requested  to  identify,  applications 
they  consider  to  be  especially  creative  or  innovative,  whether 
or  not  the  applications  were  so  identified  by  the  applicant. 

DECISION 


Concur . 


15 


RECOMMENDATION  # 51  --  (#3  page  171  of  the  GPRST  Report) 

That  IRG  members  (as  a  group  or  individually )  be  encouraged  to 
prepare  a  statement  in  addition  to  the  regular  Summary  Statement 
pointing  out  the  innovative  aspects  of  the  application  and  its 
significance.  This  statement  is  to  be  prepared  regardless  of 
whether  the  application  was  recommended  for  approval  or  disapproval ; 
these  applications  will  be  flagged  for  special  consideration  by 
BID  staff  and/or  Advisory  Council. 

DECISION 


The  effect  of  this  recommendation  would  be  to  create  a 
separate  class  of  proposals,  to  the  possible  disadvantage  of 
the  applicant.  Instead  of  preparing  a  special  report  on 
"innovative" proposals ,  Initial  Review  Group  members  should 
identify,  evaluate  and  call  attention  to  innovative  aspects  of 
an  application.  As  in  the  case  of  recommendation  49  I  do  not 
believe  the  implementation  of  recommendation  51  would  be  pro¬ 
ductive,  and  therefore  do  not  concur  with  it. 

RECOMMENDATION  052  --  (#4  page  171  of  the  GPRST  Report) 

That  NIH  consider  the  feasibility  of  developing  an  experiment 
involving  limited  support  for  certain  speculative 3  high-risk 3 
innovative  research  proposals .  Such  a  study  might  be  part  of 
a  larger 3  much-needed  effort  to  examine  the  processes  of 
decision-making  in  allocating  research  support . 

DECISION 


This  recommendation  calls  for  the  design  of  a  study  which 
at  least  would  be  extremely  difficult  if  for  no  other  reasons 
than  the  definitional  problems.  Action  should  be  deferred  for 
more  careful  consideration  and  the  development  of  the  necessary 
methodology . 

RECOMMENDATION  # 53  --  (#1  page  178  of  the  GPRST  Report) 


Recommendation  #53  was  considered  with  Recommendation  #23  on 
page  8. 
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RECOMMENDATION  # 54  --  (# 2  page  180  of  the  GPRST  Report) 

That j  when 3  following  completion  of  the  initial  technical  merit 
review  of  a  grant  application  hut  prior  to  final  action  by  a 
National  Advisory  Council /Board  3  a  BID  complies  with  a  principal 
investigator's  request  for  a  copy  of  the  pertinent  Summary 
Statement  (as  present  NIH  policy  for  implementing  the  Privacy 
Act  requires ) 3  the  BID  should  provide  the  document  with  priority 
score  displayed  and  advise  the  requestor  that  the  information 
being  transmitted  is  interim  in  nature  and  that  any  attempt3  at 
this  point  in  the  review  process  3  to  modify  the  original  application 
or  provide  commentary  for  consideration  by  the  Council /Board 
may  result  in  a  deferral  of  the  application  to  the  next  review 
cycle . 

DECISION 


Because  the  priority  score  is  only  one  element  considered 
by  the  Advisory  Council/Board,  its  release  prior  to  final  action 
could  well  be  misleading.  For  this  reason,  recommendation  54  in 
its  original  form  is  not  approved. 

The  following  rewording  is  accepted: 

"That,  when,  following  completion  of  the  initial  technical 
merit  review  of  a  grant  application  but  prior  to  final  action 
by  a  National  Advisory  Council/Board,  a  BID  complies  with  a 
principal  investigator's  request  for  a  copy  of  the  pertinent 
Summary  Statement  (as  present  NIH  policy  for  implementing  the 
Privacy  Act  requires) ,  the  BID  should  provide  the  document  without 
priority  score  displayed  and  advise  the  requestor  that  the 
information  being  transmitted  is  interim  in  nature  and  that  any 
attempt,  at  this  point  in  the  review  process,  to  modify  the 
original  application  or  provide  commentary  for  consideration 
by  the  Council/Board  may  result  in  a  deferral  of  the  application 
to  the  next  review  cycle." 

RECOMMENDATION  #65  --  (#8  page  181  of  the  GPRST  Report) 

That  NIH  should  request  authori zation 3  through  either  regulation 
or  legislation 3  as  appropriate 3  to  release  an  Initial  Review 
Group  Summary  Statement  to  the  principal  investigator  named  in 
the  application  only  after  the  review  of  his/her  grant  application 
is  complete3  i.e.3  after  review  by  National  Advisory  Council/Board. 

DECISION 


Concur . 
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RECOMMENDATION  fl 56  --  (#1  page  184  of  the  GPRST  Report) 

That  a  "single  priority  score"  convention  should  he  adopted  for 
use  throughout  NIH. 

DECISION 


This  recommendation,  if  implemented,  would  limit  the  options 
to  be  identified  in  the  study  described  immediately  below. 
Therefore  action  will  be  deferred  on  recommendation  56  pending 
completion  of  the  study  proposed  in  recommendation  57. 

RECOMMENDATION  # 57  --  (#2  page  184  of  the  GPRST  Report ) 

Thatj  before  adopting  a  single  priority  score  notation  system  for 
use  by  all  BIDs3  the  NIH  should  conduct  a  study  of  BID  practices 
regarding  the  use  of  the  "raw"  and  "normalized"  priority  scores3 
to  determine  whether  the  uniform  NIH-wide  convention  should  be 
the  use  of  the  raw  score  exclusively  or  the  present  normalized 
scores 3  whenever  they  are  available  (or  otherwise  the  raw  scores ) 3 
or  the  development  of  a  new  procedure  for  computing 3  representing , 
and/or  adjusting  priority  scores  to  compensate  for  differences  in 
group  rating  behavior . 

DECISION 


Concur . 

RECOMMENDATION  # 58  --  0)3  page  185  of  the  GPRST  Report) 

That3  until  such  time  as  a  single  NIH-wide  priority  score  notation 
system  is  adopted3  all  copies  of  Summary  Statements  which  a  given 
BID  sends  to  principal  investigators  should  display  either  the 
raw  priority  score  or  the  normalized  priority  score  (if  available) 
but  not  both 3  depending  upon  which  of  the  two  conventions  the 
particular  BID  follows. 

DECISION 

Concur.  Whichever  priority  score  (raw  or  normalized)  is  used 
by  the  Advisory  Council/Board  in  making  its  decision  on  an  applic¬ 
ation  should  be  made  available  to  the  Principal  Investigator. 
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RECOMMENDATION  ft  5  9  --  (#4a  page  185  of  the  GPRST  Report) 

NIH  should  conduct  studies  of  variations  in  individual  reviewer 
and  review  group  behavior  in  rating  applications 3  over  time 3 
and  among  different  IRGs3  and  of  the  factors  which  act  to  increase 
or  decrease  such  variability . 

DECISION 


Because  of  uncertainty  as  to  the  feasibility  and  adequacy 
of  methodology  for  such  a  study,  action  will  be  deferred  on  this 
recommendation  pending  further  discussion. 

RECOMMENDATION  ft 60  --  (ft4b  page  185  of  the  GPRST  Report) 

NIH  should  conduct  studies  of  variations  in  the  quality  of  grant 
applications  assigned  to  a  given  IRG  from  one  review  round  to 
the  next3  over  time 3  and  variations  in  the  quality  of  grant 
applications  assigned  to  different  IRGs . 

DECISION 


Serious  doubt  exists  as  to  the  adequacy  of  means  for  measuring 
the  quality  of  applications.  Action  will  be  deferred  on  this 
recommendation  pending  further  discussion  and  study. 

RECOMMENDATION  #61  --  (ft4c  page  185  of  the  GPRST  Report) 

NIH  should  conduct  studies  of  the  effects  on  the  review  process 
of  displaying  the  raw  priority  score  to  the  Initial  Review  Group 
members  immediately  after  they  assign  their  individual  scientific 
merit  ratings  3  and  of  giving  them  the  option3  at  that  point3  to 
reopen  discussion  and  rerate  the  application. 

DECISION 


Concur . 

The  following  eight  recommendations  were  considered  together: 

RECOMMENDATION  #62  --  (#1  page  193  of  the  GPRST  Report) 

That  a  formalized  NIH  grants  peer  review  appeals  system  be  established 
to  correct  or  eliminate  the  deficiencies  described  above. 

RECOMMENDATION  ft 63  --  (#2  page  194  of  the  GPRST  Report) 

That3  in  addition  to  improvements  in  current  processes  for 
reconsideration  of  assignments  or  reviews  3  certain  entirely  new 
mechanisms  be  established  for  both  types  of  appeal  to  handle 
situations  which  demand  action  or  decision  outside  the  original 
NIH  components  and  at  a  higher  level. 
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RECOMMENDATION  # 64  --  (#2a  page  194  of  the  GPRST  Report) 

That 3  central  to  these  new  mechanisms  he  the  establishment  of  the 
position  of  Ombudsman  appointed  by  the  Director 3  NIB. 

RECOMMENDATION  $ 65  --  (#2ba  page  194  of  the  GPRST  Report) 

That3  to  provide  the  needed  high  levels  of  review  related  to 

appeals 3  there  be  established :  a  Grants  Peer  Review  Appeals 
Board  (GPRAB) 3  a  permanent  committee3  the  members  of  which  would 
act  in  matters  concerning  assignment  of  grant  applications  and 
those  involving  reconsideration  of  scientific  review  of  applications . 

RECOMMENDATION  #66  --  (#2bb  page  195  of  the  GPRST  Report) 

That 3  to  provide  the  needed  high  levels  of  review  related  to 

appeals 3  there  be  established :  Grants  Peer  Review  Appeals  Panels 
(GPRAP)3  temporary  groups  which  would  be  formed  to  act  in  the 
final  stages  of  disputes  concerning  the  scientific  review  of 
grant  applications . 

RRECOMMENDATION  # 67  --  (#3  page  195  of  the  GPRST  Report) 

That  the  following  criteria  be  established  for  reconsideration 
(appeal)  of  NIB  action  on  grant  applications : 

RECOMMENDATION  #68  --  (# 4  page  196  of  the  GPRST  Report) 

That  mechanisms  and  procedures  be  established  for  appeals  concerning 
assignment  of  grant  applications . 

RECOMMENDATION  # 69  --  (#5  page  197  of  the  GPRST  Report) 

That  mechanisms  and  procedures  be  established  for  appeals  concerning 
scientific  review  of  grant  applications . 

DECISION 


This  set  of  recommendations,  all  concerned  with  the  establish¬ 
ment  of  a  formalized  grants  appeals  system,  requires  further 
discussion  and  study.  We  must  determine  the  legal,  fiscal, 
personnel  and  other  implications  of  adopting  the  recommendations. 
Immediate  consideration  should  be  given  to  formalizing  the 
present  informal  appeals  system  and  making  its  function  and 
operation  widely  known. 


February  8,  1978 


JOSEPH 
Hi  G  MS 
OD , NOH 
BL  DG  *5 


G  PEHPICH 

,  R»  137 


029059 


